Restore UNIQUE_CHECKS mysql variable to its original value when done … #32286

hostep · 2021-02-25T11:46:37Z

…with changing the database structure. This prevents potentially inserting duplicated values while manipulating data in the setup:upgrade command.

Description (*)

See #32283
When approved, please please please consider backporting this to Magento 2.3.7, this is too important to be ignored for 2.3 in my opinion.

According to Magento core devs I spoke to on Slack in the past few days, there is an underlying problem where connections should be closed and opened again between the schema manipulation and the data manipulation in order to reset these mysql variables. But that's not happening.
So this is only a workaround, but since the same workaround was implemented for the FOREIGN_KEY_CHECKS and others, it makes sense to also do this for UNIQUE_CHECKS until the root cause is found, since it's an easy fix which should not cause issues.

I have no intention to write automated tests for this change, I already spend more then 10 hours on dealing with the consequences of this issue and then an hour on finding an appropriate solution, I can't invest more time in adding automated tests, sorry!

Related Pull Requests

Fixed Issues (if relevant)

Fixes Mysql UNIQUE_CHECKS is disabled for data manipulation while it should only be disabled for schema manipulations #32283: Mysql UNIQUE_CHECKS is disabled for data manipulation while it should only be disabled for schema manipulations
Fixes Attribute value was duplicated after upgrade magento from 2.1 to 2.3 #28326: Attribute value was duplicated after upgrade magento from 2.1 to 2.3

Manual testing scenarios (*)

In some random RecurringData setup script, add some debugging code to verify the value of the UNIQUE_CHECKS variable, the RecurringData script makes it easy to test. For example, in the Magento\SalesSequence\Setup\RecurringData::install method, add:

        $connection = $setup->getConnection();
        var_dump('*** In RecurringData', $connection->query('SHOW VARIABLES LIKE "UNIQUE_CHECKS"')->fetch());

Run bin/magento setup:upgrade

Before this PR, it outputs:

string(20) "*** In RecurringData"
array(2) {
  ["Variable_name"]=>
  string(13) "unique_checks"
  ["Value"]=>
  string(3) "OFF"
}

After this PR, it outputs:

string(20) "*** In RecurringData"
array(2) {
  ["Variable_name"]=>
  string(13) "unique_checks"
  ["Value"]=>
  string(2) "ON"
}

Questions or comments

Contribution checklist (*)

Pull request has a meaningful description of its purpose
All commits are accompanied by meaningful commit messages
All new or changed code is covered with unit/integration tests (if applicable)
All automated tests passed successfully (all builds are green)

m2-assistant · 2021-02-25T11:46:40Z

Hi @hostep. Thank you for your contribution
Here are some useful tips how you can test your changes using Magento test environment.
Add the comment under your pull request to deploy test or vanilla Magento instance:

@magento give me test instance - deploy test instance based on PR changes
@magento give me 2.4-develop instance - deploy vanilla Magento instance

❗ Automated tests can be triggered manually with an appropriate comment:

@magento run all tests - run or re-run all required tests against the PR changes
@magento run <test-build(s)> - run or re-run specific test build(s)
For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names. Allowed build names are:

Database Compare
Functional Tests CE
Functional Tests EE,
Functional Tests B2B
Integration Tests
Magento Health Index
Sample Data Tests CE
Sample Data Tests EE
Sample Data Tests B2B
Static Tests
Unit Tests
WebAPI Tests
Semantic Version Checker

You can find more information about the builds here

ℹ️ Please run only needed test builds instead of all when developing. Please run all test builds before sending your PR for review.

For more details, please, review the Magento Contributor Guide documentation.

⚠️ According to the Magento Contribution requirements, all Pull Requests must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of Pull Requests happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

🎥 You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

hostep · 2021-02-25T11:47:13Z

@magento run all tests

…with changing the database structure. This prevents potentially inserting duplicated values while manipulating data in the setup:upgrade command.

hostep · 2021-02-25T12:55:05Z

@magento run all tests

magento-engcom-team · 2021-02-26T09:16:06Z

Hi @ihor-sviziev, thank you for the review.
ENGCOM-8825 has been created to process this Pull Request
✳️ @ihor-sviziev, could you please add one of the following labels to the Pull Request?

Label	Description
`Auto-Tests: Covered`	All changes in Pull Request is covered by auto-tests
`Auto-Tests: Not Covered`	Changes in Pull Request requires coverage by auto-tests
`Auto-Tests: Not Required`	Changes in Pull Request does not require coverage by auto-tests

ihor-sviziev · 2021-03-02T15:13:39Z

@magento run Static Tests

ihor-sviziev · 2021-03-02T15:14:45Z

@sivaschenko @sidolov @gabrieldagama, it seems like p1 priority should be set for this issue. What do you think?

Note: it would be great to coved this PR with some integration/setup-integration test, but not sure if it will be easy

sivaschenko · 2021-03-02T17:34:17Z

Milestone set to 2.5 as this change is backwards incompatible.

Is it possible to cover the issue with integration tests (i.e. adding a test module trying to install duplicated data)?

sivaschenko · 2021-04-12T11:02:56Z

@hostep approval is in progress

ihor-sviziev · 2021-05-11T11:42:04Z

I moved back to "to approve" status.
@sivaschenko, are there any updates on approval?

hostep · 2021-06-15T16:51:28Z

Hi folks, asking again after 2 months. PR has prio 1, still no movement, any idea what's blocking this and can I help out somehow?

ihor-sviziev · 2021-06-15T17:19:51Z

@gabrieldagama @sivaschenko @sidolov could you help with it?

mebbopakko · 2021-07-29T15:33:06Z

Any update regarding this PR?
I encountered the problem in an upgrade from 2.1.0 to 2.4.2-p1 and fortunately solved it with @hostep solution.

We have now more and more clients coming with upgrade requests and I think this should be fixed asap.

ihor-sviziev · 2021-07-29T15:41:20Z

@kandy maybe you can move forward approval for such important fix?

kandy · 2021-07-29T16:01:00Z

Unfortunately, I'm not part of the Community Engineering team. I think the best person to ping is @sidolov.
For me all looks pretty straightforward

magento-engcom-team · 2021-07-29T19:26:49Z

Hi @Den4ik, thank you for the review.
ENGCOM-8825 has been created to process this Pull Request

…ue when done … #32286

m2-assistant · 2021-08-20T15:27:43Z

Hi @hostep, thank you for your contribution!
Please, complete Contribution Survey, it will take less than a minute.
Your feedback will help us to improve contribution process.

ihor-sviziev · 2021-08-20T18:15:45Z

Finally, it got merged! 🎉

magento-engcom-team added Component: Setup Release Line: 2.4 labels Feb 25, 2021

m2-community-project bot added this to Pending Review in Pull Requests Dashboard Feb 25, 2021

m2-community-project bot added the Progress: pending review label Feb 25, 2021

Restore UNIQUE_CHECKS mysql variable to its original value when done …

2979876

…with changing the database structure. This prevents potentially inserting duplicated values while manipulating data in the setup:upgrade command.

hostep force-pushed the fix-for-issue-32283 branch from a25c2c8 to 2979876 Compare February 25, 2021 12:54

mrtuvn mentioned this pull request Feb 26, 2021

How to re install missing foreign-key constraints? #32289

Closed

5 tasks

ihor-sviziev added the Severity: S1 Affects critical data or functionality and forces users to employ a workaround. label Feb 26, 2021

ihor-sviziev approved these changes Feb 26, 2021

View reviewed changes

m2-community-project bot assigned ihor-sviziev Feb 26, 2021

m2-community-project bot moved this from Pending Review to Ready for Testing in Pull Requests Dashboard Feb 26, 2021

m2-community-project bot added Progress: ready for testing and removed Progress: pending review labels Feb 26, 2021

ihor-sviziev added Award: bug fix Award: category of expertise labels Feb 26, 2021

ihor-sviziev added the Auto-Tests: Not Required Changes in Pull Request does not require coverage by auto-tests label Feb 26, 2021

sidolov added the Priority: P1 Once P0 defects have been fixed, a defect having this priority is the next candidate for fixing. label Mar 2, 2021

m2-community-project bot added this to Ready for Testing in High Priority Pull Requests Dashboard Mar 2, 2021

m2-community-project bot removed this from Ready for Testing in Pull Requests Dashboard Mar 2, 2021

sivaschenko added this to the 2.5 milestone Mar 2, 2021

m2-community-project bot added this to Ready for Testing in 2.5 Milestone PRs Dashboard Mar 2, 2021

m2-community-project bot removed this from Ready for Testing in High Priority Pull Requests Dashboard Mar 2, 2021

sivaschenko removed the Auto-Tests: Not Required Changes in Pull Request does not require coverage by auto-tests label Mar 2, 2021

ihor-sviziev added the Progress: to approve label May 11, 2021

m2-community-project bot moved this from Ready for Testing to To Approve in High Priority Pull Requests Dashboard May 11, 2021

m2-community-project bot removed the Progress: ready for testing label May 11, 2021

Den4ik self-requested a review June 15, 2021 20:15

Den4ik approved these changes Jul 29, 2021

View reviewed changes

m2-community-project bot assigned Den4ik Jul 29, 2021

m2-community-project bot moved this from To Approve to Ready for Testing in High Priority Pull Requests Dashboard Jul 29, 2021

m2-community-project bot added Progress: ready for testing and removed Progress: to approve labels Jul 29, 2021

engcom-Hotel self-assigned this Aug 11, 2021

engcom-Hotel moved this from Ready for Testing to Testing in Progress in High Priority Pull Requests Dashboard Aug 11, 2021

m2-community-project bot added Progress: testing in progress and removed Progress: ready for testing labels Aug 11, 2021

gabrieldagama moved this from Testing in Progress to Merge in Progress in High Priority Pull Requests Dashboard Aug 11, 2021

m2-community-project bot added Progress: accept and removed Progress: testing in progress labels Aug 11, 2021

magento-engcom-team pushed a commit that referenced this pull request Aug 20, 2021

ENGCOM-8825: Restore UNIQUE_CHECKS mysql variable to its original val…

f3a8391

…ue when done … #32286

magento-engcom-team merged commit 0931e3a into magento:2.4-develop Aug 20, 2021

gabrieldagama moved this from Merge in Progress to Recently Merged in High Priority Pull Requests Dashboard Aug 20, 2021

magento-engcom-team removed this from Recently Merged in High Priority Pull Requests Dashboard Oct 14, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Restore UNIQUE_CHECKS mysql variable to its original value when done … #32286

Restore UNIQUE_CHECKS mysql variable to its original value when done … #32286

hostep commented Feb 25, 2021 •

edited

Loading

m2-assistant bot commented Feb 25, 2021

hostep commented Feb 25, 2021

hostep commented Feb 25, 2021

magento-engcom-team commented Feb 26, 2021

ihor-sviziev commented Mar 2, 2021

ihor-sviziev commented Mar 2, 2021 •

edited

Loading

sivaschenko commented Mar 2, 2021

sivaschenko commented Apr 12, 2021

ihor-sviziev commented May 11, 2021 •

edited

Loading

hostep commented Jun 15, 2021

ihor-sviziev commented Jun 15, 2021

mebbopakko commented Jul 29, 2021

ihor-sviziev commented Jul 29, 2021

kandy commented Jul 29, 2021

magento-engcom-team commented Jul 29, 2021

m2-assistant bot commented Aug 20, 2021

ihor-sviziev commented Aug 20, 2021

Restore UNIQUE_CHECKS mysql variable to its original value when done … #32286

Restore UNIQUE_CHECKS mysql variable to its original value when done … #32286

Conversation

hostep commented Feb 25, 2021 • edited Loading

Description (*)

Related Pull Requests

Fixed Issues (if relevant)

Manual testing scenarios (*)

Questions or comments

Contribution checklist (*)

m2-assistant bot commented Feb 25, 2021

hostep commented Feb 25, 2021

hostep commented Feb 25, 2021

magento-engcom-team commented Feb 26, 2021

ihor-sviziev commented Mar 2, 2021

ihor-sviziev commented Mar 2, 2021 • edited Loading

sivaschenko commented Mar 2, 2021

sivaschenko commented Apr 12, 2021

ihor-sviziev commented May 11, 2021 • edited Loading

hostep commented Jun 15, 2021

ihor-sviziev commented Jun 15, 2021

mebbopakko commented Jul 29, 2021

ihor-sviziev commented Jul 29, 2021

kandy commented Jul 29, 2021

magento-engcom-team commented Jul 29, 2021

m2-assistant bot commented Aug 20, 2021

ihor-sviziev commented Aug 20, 2021

hostep commented Feb 25, 2021 •

edited

Loading

ihor-sviziev commented Mar 2, 2021 •

edited

Loading

ihor-sviziev commented May 11, 2021 •

edited

Loading